In HDR image creation, multiple images are taken at different exposures and then blended in order to obtain an image with a dynamic range higher than the dynamic range of a single image. Within this procedure of blending or merging those images with smaller exposures, which contain more information in bright areas, improve the dynamic range within these bright areas of the image, and other images with higher exposure improve the dynamic range within dark areas. Therefore by blending these images of different exposure the dynamic range of the resulting image can be increased.
For merging images in HDR imaging the radiance maps of the images have to be computed, since there is a problem in combining the images because pixel values in two different images with different exposure times do not have a linear relationship due to nonlinear mapping of the scene radiance values to the pixel values of the taken image. Therefore, in order to merge the images, the radiance values of the original scene have to be determined, which have linear relationship with other exposure radiance values. For this determination the non-linear relationship has to be obtained from the different exposure images in order to then generate a transformation curve, which can be used to obtain linearized radiance maps of corresponding different exposure images.
For solving this problem that camera post processing of images is a non-linear process, which results in a non-linear relationship between two images with different exposure times, there are methods which require a huge amount of processing, so that they are unsuitable in practice for high bit resolutions multiple exposure images (e.g. 16 bit images).
A conventional approach, which is however not appropriate for use in HDR imaging is described in Debevec/Malik “Recovering High Dynamic Range Radiance Maps from Photographs” in ACM SIGGRAPH, August 1997.
For this method in a first step multiple photographs of a scene are taken with different exposure times Δtj. The algorithm uses these differently exposed images to recover the response function (camera response function CRF) of the imaging process of the certain camera using the assumption of reciprocity. With the so-obtained response function CRF the algorithm can fuse the multiple images into a single, HDR radiance map, the pixels of which are proportional to the true radiance values in the original scene.
This algorithm determines an inverse function of the CRF the latter mapping the actual exposure values to the pixel values. After applying this inverse function the actual radiance map is obtained. However, to determine this inverse function a quadratic objective function has to be minimized by solving overdetermined systems of linear equations. For high bit resolutions images this computational complexity is possible however too time-expensive for practical use. For example: To determine the function from five images of 16 bit resolution a system of linear equations of the order 147455 has to be solved using singular value decomposition (SVD) which is almost impractical. So for 16 bit images the Debevec-algorithm runs out of memory even on a server machine Intel® Xeon® CPU 5160, 3.00 GHz, that has 32 GB of total memory. The algorithm requires declaring a matrix of 9.02×1010 Bytes, which results into running out of memory. Even if it were possible to allocate memory, the algorithm would take approximately 75 days to solve the equation.
Therefore, it is an object of the invention to provide a method for computation of the HDR radiance map within acceptable time.
The object is solved by computation of a transformation curve (inverse camera response function) which transforms the exposure images to their corresponding radiance maps. The method according to the present invention requires only few seconds to find the transformation curve which is appropriate for transforming the images with different exposures to their corresponding radiance maps.
For this computation of the radiance maps the non linear relationship between the images taken at multiple exposures has to be represented by mean difference curves which are also determined in the course of a “Method And Unit For Generation High Dynamic Range Image And Video Frame” which is the subject of a European Patent Application filed by the same applicant on Mar. 31, 2009. Such mean difference curves relate image radiance at the image plane to the measured intensity values.